Performance of two rapid antigen tests against SARS-CoV-2 in neighborhoods of socioeconomic vulnerability from a middle-income country

Background As new and improved antigen-detecting rapid diagnostic tests for SARS-CoV-2 infection (Ag-RDT) continue to be developed, assessing their diagnostic performance is necessary to increase test options with accurate and rapid diagnostic capacity especially in resource-constrained settings. This study aimed to assess the performance of two Ag-RDTs in a population-based study. Methods We conducted a diagnostic accuracy study in neighborhoods with high socioeconomic vulnerability in Salvador-Brazil, including individuals aged ≥12 years old who attended primary health services, between July and December 2022, with COVID-19 symptoms or who had been in contact with a confirmed case. Two Ag-RDTs were compared in parallel using reverse transcription polymerase chain reaction (RT-PCR) as reference standard, the PanbioTM COVID-19 Ag test (Abbott®) and Immuno-Rapid COVID-19 Ag (WAMA Diagnostic®). Sensitivity, specificity, positive (PPV) and negative predictive values (NPV) were calculated. Results For the Abbott test the sensitivity was 52.7% (95% CI: 44.3% - 61.0%), specificity 100% (95% CI: 98.7% - 100%), PPV 100% (95% CI: 95.4% - 100%) and NPV 80.4% (95% CI: 75.9% - 84.4%). For the WAMA test, the sensitivity was 53.4% (95% CI: 45.0% - 61.6%), specificity 100% (95% CI: 98.7% - 100%), PPV 100% (95% CI: 95.4% - 100%) and NPV 80.7% (95% CI: 76.2% - 84.6%). Sensitivity for the group with Cycle Threshold (CT) <24 was 82.3% (95%CI: 72.1–90.0, n = 83) for PanbioTM COVID-19 Ag test and 87.3% (95%CI: 77.9–93.8, n = 83) for Immuno-Rapid COVID-19 Ag test. Conclusion Sensitivity for both Ag-RDT was lower than reported by manufacturers. In the stratified analysis, sensitivity was higher among those with lower CT values <24. Specificity was high for both rapid antigen tests. Both Ag-RDT showed to be useful for rapid diagnostic of potential cases of COVID-19. Negative results must be assessed carefully according to clinical and epidemiological information.


Introduction
COVID-19 has become an established and ongoing health issue.In May 2023, after more than three years since the first case of SARS-CoV-2 was identified, the World Health Organization (WHO) declared that COVID-19 is no longer public health emergency of international concern [1].However, it continues to be a global threat, especially for low-and middle-income countries where the inequities for access to testing, vaccines, and treatments maintain significant burdens and the emergence of new variants also represents a risk for further surges in cases and deaths [2,3].Consequently, early diagnoses continue to be fundamental to control the spread of SARS-CoV-2 infection and to prevent new outbreaks.WHO recommendations include mass testing and strategies that expand access to testing for COVID-19 [4].
The identification of viral RNA using real-time reverse transcription polymerase chain reaction (RT-qPCR) was commonly used as a reference standard for diagnosing SARS-CoV-2 infection, especially during the first wave of the pandemic [5].However, its expanded implementation has been hindered by its cost, the requirement of trained staff, laboratory facilities, and the time it takes to deliver results that delays self-isolation and undermines efforts to contain the disease within communities where early diagnosis to prevent the infection spread is fundamental [6,7] The early development and implementation of antigen-detecting rapid diagnostic tests for SARS-CoV-2 infection (Ag-RDT) became a fast alternative for diagnosis of this infection, since they are easy to perform, requiring minimal training and yielding results within 15 to 30 minutes [8].Ag-RDTs have demonstrated their potential to serve as a point-ofcare testing option, thus enhancing accessibility and mass testing to improve surveillance, identification of high-risk populations and outbreaks control, particularly within resourcelimited settings [9].The disparity in access to essential tools for mitigating the impact of COVID-19 was evident among low-and middle-income countries.Locally manufactured Ag-RDTs represent a potential advantage, offering an opportunity to enhance accessibility [3].
The analysis of the performance of Ag-RDT for clinical and epidemiological validation was constantly evaluated in different settings since the release of the tests at the first global COVID-19 wave [10], as such studies are essential for adopting these technologies by public health systems.A series of systematic reviews have been conducted to summarize that evidence, observing that sensitivity varied considerably between studies, with consistently high specificities; sensitivity is higher among symptomatic than in asymptomatic individuals with a pooled estimation of 73% vs 54.7% [10].In Brazil, the accuracy of some of commercialized Ag-RDTs have been evaluated with results of sensitivity lower than reported by its manufacturer varying from 9.8% to 81.1% [11].As new and improved Ag-RDTs continue to be developed, it is necessary to assess their performance to increase test options with accurate and rapid diagnostic capacity especially in resource-limited settings.In this sense, this study aimed to assess the performance of two Ag-RDTs in a population-based study.

Study design, participants and setting
We conducted a diagnostic accuracy study with data from the "Expansion of testing, quarantine, e-health, and telemonitoring strategies to fight against the COVID-19 pandemic in Brazil (TQT-COVID-19 Study)", a study conducted in high socioeconomic vulnerability neighborhoods from two Brazilian capital cities.The study protocol has been described elsewhere [12].
For this sub-study, we consecutively enrolled individuals that attended two primary health care services in Salvador between July 11 and December 5, 2022.The participants had to meet the following criteria: age � 12 years, with COVID-19 symptoms (Table 1) or who had been in contact with a confirmed COVID-19 case, regardless of previous vaccination or infection status.Individuals with active nose bleeding, acute facial injuries or trauma were not included in the study.

Data collection
Two Ag-RDTs approved by the Brazilian Health Regulatory Agency (ANVISA) were compared in parallel: the Panbio TM COVID-19 Ag test (Abbott1) and Immuno-Rapid COVID-19 Ag (WAMA Diagnostic1).Characteristics of the tests provided by the manufacturer are displayed in Table 2. Sample collection was performed prospectively by health professionals following manufacturer instructions.We used RT-qPCR testing as reference standard.Sample collection for both RT-qPCR and Ag-RDTs was carried out on the same day.One nostril was assigned for the Panbio TM COVID-19 Ag test, another nostril for the Immuno-Rapid COVID-19 Ag test, and both nostrils for RT-qPCR, consistently following the same order.
For RT-qPCR, the swab sample was immersed in an RNA shield medium (Zymo Research) to transport and contain it until extraction.Samples were transported to the reference laboratory on the same day.RNA extraction was performed using the Maxwell1 RSC 48 Viral Total Nucleic Acid Purification System (Promega).RT-qPCR reactions were performed using the Allplex SARS-CoV-2 kit (Seegene) according to the manufacturer's instructions.The RT-qPCR was considered positive when the Cycle threshold (CT) values for E-gene, RdRP-gene, and N-gene were � 40 or the CT values for two genes were under 40.The RT-qPCR was

Sample size
Using the sample size calculation proposed by Hajian-Tilaki [13] a sample size of 246 individuals would provide 80% of statistical power with a 95% confidence level, considering a sensitivity of 80% reported in a previous systematic review for individuals in the first week of symptoms [10], an expected marginal error of 0.10 and a positive rate of 26% during the TQT study.

Data analysis
The main characteristics of the population are presented using descriptive statistics.Sensitivity, specificity, positive (PPV) and negative predictive values (NPV) were calculated for each Ag-RDT compared with RT-qPCR with their corresponding 95% confidence interval (95% CI).CT values are inversely proportionate to viral load, with higher CT values indicating lower viral loads, also correlated with the infection's duration [14,15].Complementary analyses were performed comparing sensitivity, specificity, PPV, and NPV according to stratified CT values of the N-gene CT (< 24, 24 � CT � 30, and > 30), as both Ag-RDTs detect the N protein.CT categories were selected based on descriptive statistics of CT values (S1

Ethical issues
The TQT-COVID-19 study was conducted according to the Brazilian (Resolution CNS no.466, Brazil, 2012 and Resolution CNS no.510, Brazil, 2016) and international research ethics guidelines.The study was approved by the Ethics Research Committees of the World Health Organization (Protocol ID: CERC.0128A and CERC.0128B) and for local Brazilian Institutional Review Boards in Salvador (ISC/UFBA: 53844121.4.1001.5030)A written informed consent (WIC) was obtained from participants aged � 18 years.For those < 18 years a written assented consent was fulfilled by them and a WIC from parents or guardians was obtained.

Population characteristics
A total of 1459 individuals were tested for COVID-19 during the study period with at least one of the study tests.From these, 461 individuals accepted to participate in this sub-study and collected samples.We excluded 24 participants due to: RT-qPCR missing information (n = 3), inconclusive RT-qPCR result (n = 20) and indeterminate result for Immuno-Rapid COVID-19 Ag (n = 2).Thus, we included 436 individuals in this sub-study (Fig 1).The median age was 41.1 years (range 16 to 91 years), most were cisgender women (68.3%) and Pardo (44.0%).More than half of the participants had a contact with a confirmed case of COVID-19 in the previous two weeks (59.1%).The most common symptoms were headache (57.6%), dry cough (44.0%), sore throat (46.1%), fever (29.3%) and nasal congestion (30.9%).Most participants had been boosted for COVID-19 vaccines (77.5%) (Table 3).The prevalence of SARS-CoV-2 infection estimated by RT-qPCR in this sub-study was 33.9% (148/436), and the mean of CT among positives was 25.1 (S1

Discussion
We assessed the performance of two Ag-RDTs among individuals with access to two primary healthcare centers located in highly vulnerable neighborhoods in Salvador.We observed that the overall sensitivity was low for both rapid antigen tests, and lower than reported by manufacturers.However, in the stratified analysis, sensitivity was higher among those with CT values <24 and lower among those with CT values above or equal to 24.Specificity was high for both rapid antigen tests.It is worth noting that CT value differences and sensitivity variations are associated with viral load, which tends to be high during the first week of infection.Besides, both Ag-RDTs utilize the virus's nucleocapsid protein (N) as the target.This protein is commonly chosen due to its minimal variation, which could otherwise decrease sensitivity in the presence of mutations [7].
For Panbio TM COVID-19 Ag test our findings were similar to those observed in previous studies.A study conducted in Brazil included individuals with moderate or mild COVID-19 symptoms who were admitted in tertiary hospitals found a sensitivity of 60.0% (95%CI: 45.9-73.0%),with an increase in sensitivity among patients within seven days since the onset of symptoms [11].A study conducted in Kenya found a sensitivity of 46.6% (95%CI: 42.4-50.9%),with higher sensitivity among symptomatic individuals (60.6%), among who's onset of symptoms was < 5 days (67.8%) and among symptomatic individuals with CT values equal or lower than 30 (87.0%) [16].Finally, a systematic review found 24 studies assessing the mentioned test with a pooled sensitivity of 74.8% (95%CI 67.6-80.8)among symptomatic and 14 studies reporting sensitivities among asymptomatic with a pooled estimate of 56.9% (95% CI: 42.8-69.9)[10].
Regarding Immuno-Rapid COVID-19 Ag test, to the best of our knowledge, this is the first study assessing its performance in a point of care testing at community-based health services.The performance was similar to Panbio TM COVID-19 Ag test in comparison with RT-PCR.[17].This study has some potential limitations.First, we were unable to conduct an analysis stratified by presence of symptoms or by the number of days of symptoms due to the high number of missing data in both variables.However, we were able to conduct the analysis using CT values.Since our study was conducted in 2022, a period in which the Omicron variant and its sub-variants were circulating in Salvador, our results must be interpreted with caution, as there is still limited knowledge about changes in the sensitivity of tests with Omicron versus other variants.Nonetheless, a study carried out in the UK found that the performance of rapid antigen test devices was similar in different scenarios of vaccination and regarding the viral variants [18].
In conclusion, we demonstrated that a locally manufactured test with lower costs exhibited comparable performance to a test that has been in use since the initial year of the COVID-19 pandemic and against RT-qPCR.This test offers a viable alternative in resource-limited settings and holds significance in environments lacking proper laboratory facilities for COVID-19 diagnosis.Given the persistent public health concern posed by COVID-19 and the role of testing in identifying outbreaks, the adoption of accessible and effective tests should be considered while acknowledging their limitations.

Table 1 . COVID-19 symptoms considered in the TQT COVID-19 study. July
to December 2022, Salvador-Brazil.theamplification signal was absent.Any other RT-qPCR result was considered inconclusive.A socio-behavioral questionnaire was applied to individuals who accepted to participate in the study, and collected data on age, gender identity (i.e., cisgender men, cisgender women, transgender men, and transgender women), race (i.e., Black, Pardo (mixed race/skin color), Asian, Indigenous, and White), onset of symptoms, COVID-19 vaccination, knowledge, attitudes and practices for COVID-19 prevention.
• Loss of taste https://doi.org/10.1371/journal.pone.0298579.t001considered negative when Table and S1 Fig), and indeterminate CT values for the N gene, were excluded in the stratified analysis.Statistical analysis was conducted using the R project (https://www.r-project.org/)

Table 3 . Characteristics of participants included in the TQT COVID-19 study.
July to December 2022, Salvador-Brazil.results for N-gene were excluded from the stratified analysis.Additionally, 8 participants with missing data for N-gene CT values were excluded from the analysis.In the analysis stratified by CT values, for the group with CT < 24 the sensitivity was 82.3% (95% CI: 72.1-90.0,n = 83) for Panbio TM COVID-19 Ag test and 87.3% (95%CI: 77.9-93.8,n = 83) for Immuno-Rapid COVID-19 Ag test.The sensitivity was lower for the other CT value groups (Table

Table 3 .
(Continued)Overall Immuno-Rapid COVID-19 Ag test detected one case more than Panbio TM COVID-19 Ag test.While overall trends in sensitivity were similar for both rapid tests when stratified by CT category, the Immuno-Rapid COVID-19 Ag test displayed slightly higher sensitivity in the CT<24 category compared with PanbioTM COVID-19 Ag test, and the inverse was true for CT categories >24.Although both tests exhibited performance levels lower than those recommended by WHO, they still fell within the average range of available tests

Table 4 . Performance of rapid antigen tests against SARS-CoV-2 overall and stratified by N-gene CT value in TQT COVID-19 study.
July to December 2022, Salvador-Brazil.